Skip to content

feat(openai): let a host turn native tool calls on for a local runtime - #154

Merged
senamakel merged 1 commit into
mainfrom
local-native-tools
Nov 12, 2026
Merged

senamakel merged 1 commit into
mainfrom
local-native-tools

Conversation

@senamakel

@senamakel senamakel commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Why

Teeny's builder agent runs glm-5.3-flash on Surplus through OpenHuman's local-openai runtime. build_local_runtime_chat_model hard-codes native_tool_calling: Some(false), so the harness sends no tools and prompts the model with the JSON-in-tag text protocol instead. On Surplus that breaks. After the first call, every builder call failed with unknown tool \` (arguments: {})`, and every builder job ended "abandoned: the builder never reported back".

Native tool calling works for this model on Surplus. A host has to be able to say so.

Probes against Surplus (glm-5.3-flash, OpenAI-compatible), run on the Teeny box:

  1. No tools, model asked to emit JSON-in-tag: no tool_calls; the text stays text.
  2. tools=[shell], a normal ask: a clean native call, finish_reason=tool_calls, id call_7df8…, name shell, args {"command":"ls"}. Native tool calling works for this model on Surplus.
  3. tools sent plus "don't use native calls, write JSON-in-tag", non-stream: content came back verbatim, tool_calls null.
  4. Same as 3 with stream=true: only thinking chunks and no answer. Surplus attaches response_error: {format_error, content_validation: ["valid_think_chunk"], tool_calls_validation: [], extra_tags: ["only_thinking"]}. Surplus validates and rewrites output server-side, so the text protocol plus thinking is fragile there.

What

  • Adds build_local_runtime_chat_model_with_native_tools(..., native_tool_calling: bool), re-exported beside the existing builder. It is the same local-runtime profile (single system message, no vision, num_ctx, stream-backed invoke for local-openai) with native tool calling chosen by the host.
  • build_local_runtime_chat_model delegates to it with false, so every existing caller is unchanged.

Test

providers::openai::config::tests::local_runtime_builder_can_opt_into_native_tools

  • RED: error[E0425]: cannot find function build_local_runtime_chat_model_with_native_tools
  • GREEN: cargo test -p tinyinference-llm passes (882 lib tests plus the integration suites), and cargo clippy -p tinyinference-llm --all-targets is clean.

Consumers: tinyagents bumps this crate; OpenHuman then reads an opt-in local_ai.native_tool_calling setting and passes it here.

Summary by CodeRabbit

  • New Features
    • Local-runtime chat models can now be configured to use native tool calling. It remains disabled by default, preserving existing behavior unless explicitly enabled.
    • Enabling native tool calling does not change image input support or how system messages are handled.

build_local_runtime_chat_model always set native_tool_calling = false. That
is the right default for Ollama or LM Studio, but local-openai also fronts
hosted OpenAI-compatible gateways whose models call tools natively and whose
servers parse <tool_call> text in the model's own template. Prompted with the
JSON-in-tag protocol there, the gateway reads the JSON as the tool's name.

Add build_local_runtime_chat_model_with_native_tools, which takes the choice
from the host and keeps the rest of the local-runtime profile. The old builder
delegates to it with false, so existing callers are unchanged.

Co-authored-by: Medulla <medulla@tinyhumans.ai>
@tinysweeper

tinysweeper Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Reviewed head: ae1f09dd3664796a61a35eaeebb090e8bbc7328e

Verdict: Ready for maintainer review · Priority: none
Coverage: Reviewed 3 of 3 files.
Pull request purpose (title): feat(openai): let a…

Diagnostics and supporting details

Full summary

Pull request purpose (title): feat(openai): let a host turn native tool calls on for a local runtime. This pull request adds a host-controllable opt-in for native tool calling on the `local-openai` runtime, which previously hard-coded `native_tool_calling: Some(false)`. The motivation in the title and description is that a model served through an OpenAI-compatible gateway (glm-5.3-flash on Surplus) fails under the JSON-in-tag text protocol with `unknown tool` errors and abandoned builder jobs, while native tool calls work there. The change introduces a builder variant that takes the host's choice, keeps the existing builder's always-off behaviour by delegating with `false`, and re-exports the new function. The latest-push delta is unavailable, so nothing here describes a push-level delta.

Review snapshot

Change surface Files Review signal Count
Production 2 Active findings 0
Tests 1 Noted findings 0
Documentation 0 Resolved findings 0
Configuration 0 Pending checks/questions 0

Completeness: Complete
Test assessment: No supported feature-to-test mapping was available; this does not mean tests are absent or passed.

What changed

A new public builder `build_local_runtime_chat_model_with_native_tools` in `crates/tinyinference-llm/src/providers/openai/config.rs#pub fn build_local_runtime_chat_model(` area threads a host-supplied `native_tool_calling` flag into the config instead of the fixed `Some(false)`, leaving the rest of the local-runtime profile (single system message, no vision, `num_ctx`, stream-backed invoke) intact. The existing `build_local_runtime_chat_model` now delegates to it with `false`, so current callers are unchanged, and the new function is re-exported from `crates/tinyinference-llm/src/providers/openai/mod.rs`. A test in `crates/tinyinference-llm/src/providers/openai/config_tests.rs` exercises both the opt-in and default branches. Latest-push delta: unavailable — the review state has no trusted base-to-head diff comparison, so no push-delta content is reported.

Features

  • Added — Host-selectable native tool calling for local-runtime OpenAI-compatible models: A host that knows its endpoint supports native tool calls can turn them on for the local-runtime profile, avoiding the JSON-in-tag text protocol that the description reports failing on Surplus; every other aspect of the local-runtime profile is unchanged. (crates/tinyinference-llm/src/providers/openai/config.rs pub fn build_local_runtime_chat_model(, crates/tinyinference-llm/src/providers/openai/mod.rs)
  • Modified — `build_local_runtime_chat_model` delegates with native tool calling off: Existing callers keep the previous always-off native tool calling behaviour because the old entry point forwards `false` to the new builder. (crates/tinyinference-llm/src/providers/openai/config.rs pub fn build_local_runtime_chat_model()

Tests

No supported feature-to-test mapping was produced. Test execution is not inferred.

Findings

No active actionable findings.

Before merge

None.

Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The critique lane answered with no findings: the flag is applied through the derived profile so `with_native_tool_calling` semantics hold, and vision/single-system-message behaviour is untouched.
  • Lane summary: The change extracts a `build_local_runtime_chat_model_with_native_tools` variant that threads the host's `native_tool_calling` choice into `OpenAiConfig`, with the existing `build_local_runtime_chat_model` delegating to it with `false` so the previous always-off profile is preserved, and it re-exports the new function. The override is applied through `configure_openai_model` after the model is constructed, so it lands on the derived profile as `with_native_tool_calling` requires, and vision/single-system-message behaviour is untouched; the added test covers both the on and off paths. I found no correctness defect in the change.

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The security lane answered with no findings: no untrusted input path, credential handling, or permission change is introduced by the opt-in flag.
  • Lane summary: The change adds a `build_local_runtime_chat_model_with_native_tools` builder that lets a host opt into native tool calling for a local OpenAI-compatible endpoint, with the existing `build_local_runtime_chat_model` delegating to it with `false` so its behaviour is unchanged, plus a test and a re-export. No untrusted input path, credential handling, or permission change is introduced; the change looks safe to merge.

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The tests lane answered with no findings and characterized the added test as exercising both the `true` and `false` branches with profile assertions, while the delegating builder keeps `false` as before.
  • Lane summary: The change adds `build_local_runtime_chat_model_with_native_tools`, threads the flag through the config into `native_tool_calling`, re-exports it from `mod.rs`, and adds `local_runtime_builder_can_opt_into_native_tools`, which exercises both the `true` and `false` branches and asserts the resulting profile (tool_calling on/off, vision off, system-message handling unchanged). That is a real, failing-on-regression assertion for the only behaviour that changed; the delegate keeps `false` as before. No defect found.

commits

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The commits lane answered reporting no sensitive content in committed files.
  • Lane summary: No sensitive content in committed files.

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The description lane answered with no findings: the delegation with `false`, the re-export from `providers::openai`, and the test covering opt-in and default are consistent with the diff and the PR description, and the doc comment is accurate.
  • Lane summary: The new builder and its delegation are consistent with the diff and the PR description: `build_local_runtime_chat_model` now delegates with `native_tool_calling: false`, the new public function is re-exported from `providers::openai`, and the added test exercises both the opt-in and the default. The doc comment describes the change accurately, and no repository rule in the supplied list is violated by the diff.
Evidence and run details
  • Models: ladder/vectors-oai3, deepseek-v4.1-flash
  • Spend: $0.004247
  • Tokens: 140123 input · 12341 output · 35840 cached · 259 embedding
  • Updated: 2026-11-12T13:31:14Z
Head State Pass summary
ae1f09dd3664 ready for maintainer review 0 active finding(s), 0 resolved finding(s) (at 2026-11-12T13:31:14Z)

tinysweeper 0.1.2

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The local-runtime chat model builder now supports an explicit native tool calling option. The existing builder keeps native tool calling disabled by default. The new builder is publicly re-exported, and tests cover enabled and disabled settings.

Changes: Local-runtime builder

Layer / File(s) Summary
Configurable builder and coverage
crates/tinyinference-llm/src/providers/openai/config.rs, crates/tinyinference-llm/src/providers/openai/mod.rs, crates/tinyinference-llm/src/providers/openai/config_tests.rs
The existing builder delegates with native tool calling disabled. The new public builder accepts the setting and applies it to the local-runtime model. The provider re-exports the new builder. Tests check enabled and disabled tool calling, plus related profile settings.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Suggested reviewers: m3ga-mind, tinysweeper

Merge Risk: Minimal · up to ae1f0

This change adds an opt-in builder for native tool calling on local runtimes and leaves the existing builder's behavior unchanged. No merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: a new builder, build_local_runtime_chat_model_with_native_tools, lets the host enable native tool calling for a local runtime. The raw summary confirms the new public …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: a new builder, build_local_runtime_chat_model_with_native_tools, lets the host enable native tool calling for a local runtime. The raw summary confirms the new public function, its re-export, and a test for the opt-in. The existing builder keeps its false default, so the title matches the changeset's primary intent. The title is concise, specific, and follows the conventional-commit style.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the tools are there
Then tests the off and on affair
The old path keeps its setting clear
A new choice joins the builder here
The bunny hops with tests in gear

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b776919d-c253-4810-9adb-aa18857911bd
Commits

Reviewing files that changed from the base of the PR and between 5d96fa7 and ae1f09d.

Files selected for processing: 3

  • crates/tinyinference-llm/src/providers/openai/config.rs
  • crates/tinyinference-llm/src/providers/openai/config_tests.rs
  • crates/tinyinference-llm/src/providers/openai/mod.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T13:31:38.863899Z ae1f09d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

Diagnostics

Total spend: $0.0042

@senamakel
senamakel merged commit baedacd into main Nov 12, 2026
11 of 13 checks passed
sub4biz pushed a commit to sub4biz/tinyagents that referenced this pull request Nov 14, 2026
…ols)

Picks up build_local_runtime_chat_model_with_native_tools
(tinyhumansai/tinyinference#154) so OpenHuman can turn native tool calls on
for a local-openai endpoint that supports them.

Co-authored-by: Medulla <medulla@tinyhumans.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant